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Statistical  Cost  Estlmat 
Some  Baaie  Issues  (Aircn 


I.  Iff  gJKPOBE.  1  1  r  ' 

The  plan  of  my  presentation  is  first  to  discuss  sane  of  the  basic 
issues  of  Statistical  Cost  Estimating  Relationships  (CER's),  and  then 

/^describe:  briefly  sene  our  recent  studies  ehich  have  attempted  to 

?  ; 

develop  such  CER's  on  aircraft  programs.  I  will  conclude:  by  sunmar icing 
the  major  advantages  and  disadvantages  of  statistical  CER's. 


II.  THE  MAJOR  CHARACTERISTICS  OF  STATISTICAL  CER  TECHNIQUES. 

The  major  characteristics  of  the  statistical  CER  technique,  as  I 
am  using  the  term,  are  (as  shown  in  Figure  l): 

a)  First,  is  the  use  of  actual,  historical,  data  on  the  costs  of 
other  (both  past  and  current)  related  programs.  Actual  data,  rather 
than  proposal  data,  or  other  estimates,  are  sought. 

b)  Secondly,  these  actual  costs  are  then  related  to  selected  cosc- 
lnfluenclng  program  characteristics,  particularly  the  major  physical  and/ 
or  performance  characteristics. 


c)  Finally,  statistical  procedures  are  used  to  develop,  test,  and 
to  evaluate  the  cost  estimating  equations  -  the  CERs.  (I  am  using  the 
term  "statistical"  rather  loosely  here.  In  my  view  the  relation  that 
would  result  from  fitting  a  line,  by  eye,  to  a  fev  data  points  on  a  scatter 
dlagrms  wild  qualify  as  a  statistical  CER. ) 


/ 


A  major  assumption  in  the  use  of  statistical  CER's  is  that  the  new 
programs  on  which  the  CER's  are  to  he  used,  will  be  affected  by  the  cost 
influencing  variables  (i.e.  Independent  variables)  in  approximately 
the  sane  way  as  they  affected  the  historical  programs  used  to  derive  the 
CIR's. 

III. .  THE  POST  ESTIMATING  TASK  ADDRESSED  HERE. 

The  problem  that  we  are  attempting  to  solve  with  the  help  of  CER's 
is  that  of  estimating  the  costs  of  future  programs.  For  my  purpose  here 
I  am  excluding  the  problem  of  estimating  the  costs  of  a  follow-on  program 
That  is,  we  are  seeking  estimates  until  the  time  that  the  item  goes  into 
production  and  a  sufficient  body  of  actual  costs  on  that  item  begins  to 
be  reported. 

• 

The  use  of  CER's  to  compare  one  contractor  cost  proposal  to  another 
is  a  more  controversial  application  of  CER's. 

IV.  WHY  CER's? 

The  emphasis  on  CER's  has  arisen  because  of  numerous  examples  of 
poor  estimates  in  the  past  on  major  programs.  Estimates  in  the  past 
have  seemed  to  rely  too  heavily  cm  contractor  proposal  estimates,  which 
repeatedly  have  turned  out  to  be  considerably  underestimated.  The  recent 
efforts  in  my  office  can  be  said  to  have  been  given  primary  impetus  by 
initial  estimates  on  the  7-111  program.  Numerous  other  examples  of  major 
underestimates  such  as  the  B-70,  Skybolt,  Mk  46  torpedo,  SHRIKE  missile, 
etc.,  can  be  cited. 
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This  is  not  to  say  that  we  consider  the  statistical  CBR  as  being 
the  last  word,  or  even  the  only  word,  in  making  future  program  cost 
estimates.  The  use  of  engineering  evaluations  and  the  examination  of 
the  specific  design  details,  with  cost  comparisons  of  individual  com¬ 
ponents,  can  also  provide  considerable  insight.  These  other  techniques, 
together  with  appropriate  CER's,  should  result  in  considerably  improved 
cost  estimates.  Where  specific  equipment  designs  are  not  available, 
recourse  to  CER's  may  be  the  only  option.  This  is  often  the  case,  when 
OSD  or  the  Services  are  doing  advanced  Cost-Effectiveness  studies. 

V.  MAJOR  PROBLEMS  UT  THE  DEVELOPMENT  OF  STATISTICAL  CER's. 

for  these  who  have  not  been  through  the  experience  of  developing 
statistical  CER's,  I  would  like  to  describe  briefly  some  of  the  major 
procedural  problems  typically  encountered,  using  a  few  aircraft  examples. 
Subsequently,  I  will  describe  In  more  detail  the  specific  aircraft  studies 
sponsored  by  our  office. 

These  procedural  problems  can  be  divided  into  the  categories  shown 
in  Figure  2. 

A.  Choice  of  the  dependent  variables. 

The  choice  of  the  dependent  variable  involves  such  problems  as: 

-  whether  to  estimate  cost  per  pound  or  to  estimate  cost 
per  airframe  directly 

-  at  what  production  unit,  or  units,  to  make  the  estimate 

-  and  what  cost  categories  should  be  covered  by  each 
equation. 


•  EVALUATION  OF  PREDICTION  CAPABILITY 


Currently,  the  popular  approach  for  aircraft  airframes  la  to  estimate 


separately  manufacturing  labor,  manufacturing  materials,  tooling  and 
engineering  and  usually  on  a  coat  per  pound  basis.  Overhead,  Odd  and 
profit  rates  are  then  applied.  The  difficulty  in  separating  development 
from  production  costs  in  the  data  has  resulted  in  these  frequently  being 
estimated  together  in  airframe  CEP's.  Total  engine  and  total  avionics 
costs  are  also  each  estimated  separately.  The^e  are  also  proponents  of 
estimating  total  cost  directly,  at  least  total  airframe,  if  not  total 
flyaway  cost.  Another  aspect  of  the  dependent  variable  question  is  the 
coverage  of  the  equations  regarding  the  end  item.  Far  example,  aircraft 
sight  be  subdivided  into  various  aircraft  types  (such  as  fighters,  bombers, 
transports,  etc.),  by  weight  class,  by  speed  (e.g.,  subsonic  versus  super- 
sonic),  or  by  type  of  propulsion  system  (e.g.,  reciprocating  versus  turbo- 
Jet ). 

B.  Choice  of  the  Independent  Variables. 

The  choice  of  the  independent  variables  Is  the  problem  of  ex¬ 
tracting  the  most  important  coat  influencing  characteristics,  from  the 
literally  hundreds  of  character  1st  lea  associated  with  any  given  aircraft 
program.  It  la  worthwhile  to  review  same  of  the  ma^or  types  of  factors 
affecting  coat.  These  are  listed  in  Figure  3*  It  is  to  be  noted,  however, 
that  typically,  CEB’s  have  not  directly  addressed  some  of  these  shown  in 
Figure  3.  This  serves  to  indicate  how  complex  the  over-all  problem  of 
coat  estimation  le. 

1.  Physical  characteristics  such  as  weight,  and  site  are  found 
in  moat  current  CEB's.  They  are  used,  in  a  sense,  as  "proxy"  (substitute) 
varlaeies  for  ..... 


TIVE  QUANTITY  AND  PRODUCTION  RATE 


2.  The  performance  characteristics  such  as  speed,  range,  pay- 
load  and  reliability.  We  would  in  many  cases  prefer  to  estimate  coat 
solely  as  a  function  of  performance.  However,  typically,  CER 'a  include 
both  physical  and  performance  variables  since  certain  physical  charac¬ 
teristics  such  as  gross  take-off  weight  are  usually  available  even  In 
the  very  early  stages  of  program  concept  formulation.  An  exception  Is 
our  experimental  model  far  estimating  transport  costs  which,  as  will  be 
noted  later,  alms  at  relating  cost  directly  to  a  measure  of  over-all 
mission  performance. 

3.  Even  with  the  same  physical  »  id  performance  characteristics, 
title  costs  of  a  given  program  can  vary  due  to  the  particular  manufacturer 
—  because  of  such  factors  as  the  quality  of  the  management,  the  degree 
of  plant  modernisation,  the  plant's  other  woriuoad,  and  the  manufacturer's 
prior  experience  on  recent,  related  programs.  Moat  often,  DOD  CER  studies 
use  data  samples  containing  programs  from  several  manufacturers .  The 
resulting  CER 'a  therefore  represent  a  "weighted  average"  of  the  manufac¬ 
turers.  However,  meet  of  the  aircraft  airframe  CSS  procedures  at  least 
do  provide  provision  for  applying  individual  manufacturers'  direct  labor, 
overhead,  and  OLA  rates  for  those  programs  where  specific  manufacturers 
can  be  Identified. 

k.  Che  prices  of  labor  and  materials  can  change  doe  to  various 
external  influences.  Moat  CSR  studies  adjust  historical  data  using 
Industry  pries  indices  and  provide  CSR's  that  estimate  costa  in  constant 
dollars.  If  the  cost  estimator  wishes  to  estimate  a  new  program  In 


current  dollars,  these  CER  proeeuures  require  him  to  obtain  separately 
price  level  projections  which  would  then  be  applied  to  the  CER  program 
estimates. 

5 .  Engineering  changes,  and  changes  in  the  pace  of  the  program, 
can  affect  significantly  the  eventual  cost  of  a  program.  Their  effect 
upon  CER's  is  a  complicated  one.  This  area  offers  a  wide  open  opportunity 
for  some  Imaginative  and  practical  research, 

6.  The  quality  problem  in  CER’s  is  typically  handled  by  add¬ 
ing  a  cost  quantity  relation  to  the  basic  CER  which  estimates  cost  at  a 
specific  production  quantity.  There  also  are  a  few  studies  which  use  the 
cumulative  production  quantity  as  one  of  the  independent  variables  in  the 
basic  CER. 

The  production  rate  problem  has  frequently  not  been  addressed 
directly  in  CER's.  Planning  Research  Corporation  in  its  airframe  work 
for  our  office,  which  I  will  discuss  later,  has  included  a  production  rate 
variable  in  some  of  its  equations.  However,  none  of  us  are  yet  very  happy 
with  our  understanding  of  the  effect  of  production  rate  upon  cost.  This 
ie  another  useful  area  for  future  research, 

7.  Finally,  the  f,0ther:'  category  includes  such  effects  as  plant 
fires  or  labor  strikes.  If  these  are  truly  unique  events,  it  probably  is 
appropriate  to  adjust  the  historical  data  where  such  events  and  their 
costs  can  be  identified,  Otherwise,  the  cost  equations  will  include  the 
effects  of  such  events.  The  equations  should  not  normally  be  constructed 
uO  predict  explicitly  these  unusual  costs. 
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C.  Choice  of  the  Form  of  the  Equations. 

The  particular  form  of  the  equations  to  be  used  is  seldom  com¬ 
pletely  clear.  The  variables  can  be  added,  multiplied  together,  given 
exponents,  etc.  Figure  4  shows  two  examples  of  CEB's.  PRC  has  generally 
used  linear  combinations  of  terns  (as  in  the  first  equation),  though 
each  tern  itself  may  consist  of  various  forms  of  the  variables.  BARD 
has  usually  used  the  multiplicative  or  exponential  form  (as  shown  in  the 
second  equation).  It  is  seldom  intuitively  clear  what  form  is  most 
appropriate. 

D.  Handling  of  the  Input  Bats. 

Unfortunately,  as  most  of  you  are  probably  aware,  the  basic 
input  data  usually  has  many  deficiencies.  Appropriate  data  massaging, 
such  as  adjustments  for  different  subcontracting  policies  or  for  dif¬ 
ferent  accounting  definitions,  is  generally  a  necessary  task.  If  done 
improperly,  however,  it  can  lead  to  misleadingly  good  equation  fits  to 
the  adjusted  sample  data.  The  lost  Information  Be  porting  System  being 
developed  for  the  Department  of  Defense  deals  with  this  vital  problem 
of  adequate  data,  at  least  as  far  as  major  procurement  costs  are  con¬ 
cerned.  Incidentally,  to  add  to  our  woes,  our  office  is  beginning  to 
have  an  uneasy  feeling  that  the  phys leal  and  performance  characteristic 
data  used  in  CER  studies  may  also  be  deficient.  (We  need  to  review  this 
possibility  and  correct  the  problem  as  needed. ) 

E.  Evaluation  of  Prediction  Capability. 

In  presenting  these  major  procedural  problems  in  developing 
CER's,  I  have  said  little  about  solving  them.  Unfortunately,  I  can 


say  little  of  a  definitive  nature.  However,  It  la  probably  neither 
necessary,  nor  is  It  desirable,  to  pre-spccify  exactly  how  these  should 
be  handled  in  each  case.  Essentially,  decisions  as  to  the  variables  and 
forms  of  the  equations  have  to  be  answered  for  each  individual  study. 
Universally  applicable  solutions  are  not  likely  to  be  found.  Initially, 
on  a  study,  engineering  type  Judgments  are  used  to  make  preliminary  selec¬ 
tions.  Subsequently,  statistical  analysis  provides  the  major  tools  for 
testing  the  various  alternatives.  Even  after  this  however,  the  analyst 
may  find  it  desirable  to  modify  the  estimation  procedure  (such  as  with 
an  adjustment  for  exotic  new  airframe  materials )  in  a  manner  which  he 
feels  will  help  discriminate  future  program  costs. 

This  leads  us  to  the  key  problem  that  no  discussion  of  CEB's  or 
of  any  predictive  technique,  can  ignore.  That  is  the  question  of  Just 
how  good  is  the  procedure  for  predicting.  We  feel  that  other  techniques 
for  estimating  the  costs  of  future  programs  are  by  themselves,  not  good 
enough.  But  is  the  use  of  statistical  CER’s  any  better?  Unfortunately, 
this  is  as  yet  a  question  on  which  we  do  not  have  completely  satis¬ 
factory  evidence  though  what  we  do  have  is  encouraging.  We  currently  use 
the  following  approaches  to  attempt  to  evaluate  the  capability  of  statis¬ 
tical  CER's: 

1)  The  mam  approach  is  to  look  at  the  statistical  evidence  as 
to  how  good  an  equetion  fits  the  sample  data.  We  look  at  such  statistics 
as  the  correlation  coefficient,  and  the  standard  error  of  estimate.  The 
presumption  is  that  the  better  the  fits,  the  more  likely  are  the  equations 
to  predict  with  accuracy. 
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2)  Jocrfcbar  approach  la  to  leave  out  of  the  sample  one  or  two 
data  points  and  then  to  test  the  derived  equation  against  them.  If 
the  equation  provides  a  "reasonably"  dose  estimate,  our  confidence  in 
the  equation's  prediction  capabilities  increases;  otherwise,  our  confi¬ 
dence  decreases. 

3)  A  recent  proposal  is  to  perform  "Historical  Simulation" 
to  see  how  the  method  for  deriving  the  CER's  would  have  performed  in 
the  past.  This  is  an  interesting  idea  and  should  be  explored  further. 
(Hr  office  has  been  looking  into  this  but  unfortunately  has  not  been 
able  to  spend  much  time  on  it.)  But,  again,  historical  simulation  can 
provide  at  best  only  indirect  evidence. 

4)  The  real  predictive  test  cf  the  CER's,  or  any  prediction 
technique,  is  of  course  to  see  how  successful  they  actually  are  on  real 
programs  after  the  results  are  in.  Unfortunately,  this  usually  means  a 
long  wait.  However,  we  should,  at  least  keep  careful  histories  of  our 
estimates,  and  the  estimating  techniques  used,  to  help  eventually  to 
evaluate  how  they  performed  and  for  dues  as  to  how  they  might  be  Im¬ 
proved.  The  Air  Force  Systems  Caiman d  Tracking  System  recently  estab¬ 
lished  is  a  good  step  in  this  direction. 

VI.  SPECIFIC  OSD  AIRCRAFT  STUDIES. 

low,  I  would  like  to  describe  briefly  scorn  of  the  specific  aircraft 
studies  that  our  office  has  sponsored. 

As  noted  earlier,  the  F-Ul  cost  analysis  work,  which  began  in  our 
office  in  1963,  gave  Impetus  to  the  coat  research  on  aircraft  programs. 


Since  that  time  our  office  has  sponsored,  or  engaged  In,  a  number  of 
studies  on  aircraft  costs  Figure  5  lists  the  five  studies  that  I  will 
describe  here. 

l)  The  first  study  is  the  work  of  the  Planning  Research  Corpora¬ 
tion.  PRC  prepared  a  report  issued  last  February  (1965)*  R5^7,  "Methods 
of  Estimating  Fixed-Wing  Airframe  Costs."  It  has  been  given  vide  dis¬ 
tribution  throughout  DOD. 

The  PRC  study  utilized  a  number  of  imaginative  and  rrovocativi 
approaches.  Though  the  specific  procedure  suggested  by  PRC  for  esti¬ 
mating  airframe  costs,  seems  to  me,  on  the  whole,  to  be  overly  cumbersome 
and  involved,  it  has  served  to  stimulate  much  constructive  discussion. 

I  will  review  Just  a  few  of  the  study's  characteristics  which  are  of 
interest  here: 

a)  CER's  are  provided  for  each  cost  element:  manufacturing 
man-hours,  and  materials,  engineering,  and  tooling  costa. 

b)  About  40  Air  Force  and  Navy  aircraft  were  used  in  the  manu¬ 
facturing  man-hour  sample.  For  the  other  coat  elements,  usable  data  was 
found  for  about  25  aircraft.  The  CER's,  with  Bate  qualifications,  are 
meant  to  apply  to  all  types  of  aircraft,  e.g.,  fighters,  bombers,  etc., 
as  well  as  to  both  subsonic  and  supersonic  aircraft. 

c)  Numberous  data  adjustments  were  made  by  PRC  including  the 
application  of  'Company  Adjustment  Factors"  for  certain  companies  whose 
costs  in  the  basic  sample  appeared  to  be  generally  low  or  high.  This 
"bias"  observed  by  PRC  could  result  from  either  company  accounting  prac¬ 
tices,  from  manufacturer  efficiency,  or  both.  PRC's  description  of  the 
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data  problems  and  Its  various  data  adjustments  make  worthwhile  reading 
for  those  becoming  Involved  with  similar  problems. 

d)  PRC  utilized  a  number  of  variables,  other  than  aircraft 
physical  and  performance  characteristics,  to  account  for  certain  of  the 
other  factors  affecting  cost.  These  included  a  production  rate  variable, 
a  "weight  growth"  (or  engineering  change)  factor  variable,  and  a  "time" 
variable.  (The  time  variable  in  PRC's  analysis  showed  a  significant 
explanatory  ability.  Presumably  this  variable  reflects  such  factors  as 
improved  safety,  reliability,  crew  comfort,  etc.,  which  are  not  reflected 
by  the  other  explanatory  variables  but  which  are  correlated  with  time.) 

e)  The  coet-quantlty  relationship  is  derived  by  preparing 
statistical  OCR's  at  each  of  four  cumulative  production  quantities,  10, 

30,  100,  and  300.  For  a  given  aircraft,  a  best-fit  line  connecting 
vhese  points,  would  then  represent  the  cost-quantity  curve. 

A  second  edition  of  the  PRC  report  is  currently  being  completed. 
This  will  incorporate  some  additional  cost  data  as  well  as  attempt  to 
improve  some  of  the  techniques  such  as  a  needed  Improvement  to  its  "weight 
growth"  procedure  and  the  separation  of  recurring  from  nonrecurring  tooling 
and  engineering  coete. 

2)  As  an  outgrowth  of  the  C-5  analytes  and  the  desire  of  our  offloe 
to  have  an  independent  cost  estimate  on  this  major  program,  we  Jointly 
undertook  with  PRC  a  study  to  develop  CEP's  for  large  transport  aircraft. 
The  PRC  portion  of  this  work  is  presently  being  finalised.  It  will  be 
distributed  when  available.  This  report  will  cover  only  the  airframe. 
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Ingine  and  avionics  estimates,  however,  were  made  in-house  using  rougher 
tools.  PRC  for  the  study  drew  heavily  from  Its  work  in  R5^7«  However, 
in  this  study,  PRC  found  that  the  best  statistical  results  were  achieved 
by  utilising  ~  sample  consisting  only  of  transports  and  bombers.  The 
explanatory  variables  chosen,  such  as  payload  weight,  appear  more  appro¬ 
priate  to  the  estimation  of  large  transport  aircraft.  Also,  a  somewhat 
different  "weight  growth"  approach  was  included  in  the  final  equations. 

3)  In  1964  our  office  monitored  and  directed  for  the  Department 
of  Commerce  a  coet  analysis  study  for  the  Commercial  Supersonic  Trans¬ 
port  program.  Two  contractors.  Operations  Research  Inc.  and  PRC,  under¬ 
took  parallel  studies  to  develop  cost  estimating  relations  for  all  phases 
—  Development,  Investment,  and  Operating  —  of  the  S8T.  Unfortunately, 
at  this  time,  this  material  has  not  been  released  for  general  usage. 

However,  organizations  able  to  provide  sufficient  Justification  would  be 
able  to  review  the  material.  The  samples  used  consisted  primarily  of 
DOD  aircraft.  PPC's  procedures  were  again  similar  to  those  used  in  its 
R54?  report.  ORI,  however,  developed  equations  separately  for  subsonic 
and  supersonic  aircraft.  It  also  applied  ar.  "empirically"  developed 
titanium  coat  factor  to  adjust  for  expected  increases  in  airframe  labor, 
materials,  and  tooling  costs  due  to  the  substantial  use  of  titanium  In 
the  airframe,  both  contractors  prepared  statistical  CER's  for  engine 
costs  as  well  as  airframe  costs.  Hie  primary  explanatory  variables  used 
in  the  engine  equations  were:  thrust,  weight,  speed,  and  turbine  inlet 
temperature.  Hie  sample  sizes  for  the  engine  analyses  were  quite  small, 
from  5  to  14  engines. 
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The  operating  coet  equations  derived  for  the  study  reflect  commer- 
clal  practices,  and  though  of  saw  Interest,  are  not  fully  useful  for 
DOD  purposes. 

k.  Internally,  our  office,  as  part  of  our  technique  research  pro* 
gram,  has  experimented  on  transport  aircraft  GER's.  This  study  currently 
Is  attempting  to  relate  cost  primarily  to  an  over-all  mission  performance 
level.  In  this  case  ten -miles  per  hour  ....  and  to  estimate  total  air¬ 
craft  flyaway  cost  directly  rather  than  estimating  the  individual  compon¬ 
ents,  such  as  labor,  materials,  engineering  and  tooling  or  the  Individual 
subsystems  such  as  airframe,  engines,  and  avionics. 

5.  He  currently  are  also  undertaking  internally  a  program  to 
develop  CSR's  on  helicopter  airframe  procurement  costs.  These  CSR's 
will  cover  helicopters  of  all  three  services.  The  sample  presently  con¬ 
sists  of  approximately  13-1?  helicopters.  It  will,  of  course,  attempt  to 
taloe  Into  account  the  work  on  air  frame  man-hours  already  performed  by  the 
Any. 

6.  In  discussing  these  five  projects,  I  have  made  little  mention 
of  avionics  procurement  costs  or  of  aircraft  operating  costs.  Both  these 
areas  have  so  far  seemed  to  have  had  little  attention  paid  to  them  as  far 
as  the  development  of  CSR's  is  canoerned.  Ky  office  did  sponsor  an  abor¬ 
tive  avionics  study  last  spring.  The  recent  work  by  Nr.  Tfeng  of  SAID 
(RM  W51-PR  dated  February  1966)  which  provides  CSR's  for  aggregative 
avionics  systems  for  fighter  and  Interceptor  aircraft.  Is  the  only  pres¬ 
ently  available  significant  study  that  I  am  aware  of.  Aircraft  avionics 
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is  becoming  an  increasingly  important  element  of  cost  and  merits  greater 
research  efforts. 


On  operating  costs ,  the  SST  work  as  indicated  earlier,  though 
interesting,  was  primarily  commercially  oriented.  My  office  has  recently 
keen  trying  to  get  into  this  area.  Thus  far,  our  efforts  have  been  di¬ 
rected  primarily  at  the  data  problems. 

However,  PRC  has  performed  for  us  with  same  success  a  test  study 
on  fuel  CRR's  for  Air  Force  aircraft  using  cost  data  currently  available 
in  the  Air  Force  Planning  Factors  Manual.  The  sample  contained  l6  air¬ 
craft  of  various  types.  An  interesting  feature  of  this  study  is  Its 
presentation  of  two  equations,  one  far  use  early  in  the  planning  stages 
when  only  such  characteristics  as  speed  and  weight  are  available.  The 
other  equation  is  for  use  in  later  stages  wben  an  estimate  of  such  charac 
terlstlcs  as  engine  specific  fuel  consumption  is  also  available.  This 
report  will  be  distributed  shortly. 

Both  the  avionics  and  operating  cost  fields  are  clearly  vide 
open  for  CZR  work.  Data  problems,  as  usual,  have  probably  been  the 
chief  stumbling  block.  We  will  be  aoot  Interested  in  seeing  results 
from  the  Air  Forces'  current  CZR  program  on  OiM  costs. 

vn.  owi iwu  c?  Asggggas  MBApyAimzas  of  statistical  era's. 

Finally,  I  would  like  to  list  what  I  believe  to  be  the  major  ad¬ 


vantages  and  disadvantages,  of  statistical  CZR'a. 


A.  Advantages  (Figure  6) 

1)  First  they  penult  relatively  rapid  cost  estimation.  A 
quick  reaction  time  is  possible  after  the  appropriate  CER's  are  avail* 
able.  (The  development  of  the  CER's  themselves,  of  course,  is  a  more 
lengthy  process . ) 

2)  Second,  only  a  relatively  small  amount  of  manpower  la 
required  to  make  the  estimate  with  CER's.  (Note,  however,  that  making 
an  estimate  f c*  a  nev  program  is  not  a  trivial  exercise.  Considerable 
judgnent  is  required  in  determining  the  extent  of  applicability  of  the 
GBR  to  the  nev  program  and  In  making  adjustments,  if  any  are  required, 
to  the  CER  estimating  procedure.) 

3)  Third,  the  procedures  for  developing  statistical  CER's,  and 
for  the  subsequent  use  of  them,  is  a  relatively  objective  process. 

4)  Fourth,  the  procedure  is  a  systematic  one  which  will  pro¬ 
vide  consistent  estimates  ard  ones  which  can  be  reproduced.  This  permits 
considerable  visibility  so  that  other  persons  can  relatively  easily 
evaluate  them. 

5)  And  moat  importantly,  the  use  of  (actual  historical  data,  in 
a  systematic  manner,  seems  to  lead  to  potential  accuracy  advantages  at 
compared  to  other  current  techniques  for  estimating  costs  of  nev  systems. 

B.  Disadvantages  (Figure  7) 

1)  First,  past  practices  and  occurrences  implied  in  the  his¬ 
torical  cost  data  will  be  reflected  in  the  derived  cost  equations  and 
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therefore  in  the  estimates  for  the  new  system.  To  the  extent  that 
future  practices  will  differ,  estimation  errors  are  likely  to  occur. 
Should,  for  example,  the  Value  Engineering  program  or  incentives  from 
greater  use  of  incentive  contracting,  significantly  improve  manu¬ 
facturers'  efficiencies  in  future  programs,  CEE's  based  upon  past  prac¬ 
tices  would  tend  to  give  overestimates.  (Note,  however,  that  provision 
far  such  occurrences,  at  least  currently,  may  be  more  realistic,  and  it 
la  this  which  :elps  to  avoid  cost  over-optimism. ) 

2)  Second,  there  is  a  tendency  to  over-simplify  the  many 
factors  which  do  affect  cost.  CER's  must  as  a  practical  matter  be  re¬ 
stricted  to  a  relatively  few  explanatory  variables.  With  a  new  program, 
there  may  be  introduced  a  characteristic  of  major  cost  importance  (such 
as  an  unusually  stringent  reliability  requirement)  but  with  respect  to 
which  the  equations  do  not  discriminate  because  the  characteristic  was 
not  of  importance  in  explaining  cost  in  the  data  sample.  Human  judgment 
is  necessary  with  CEE's  to  consider  such  possibilities  and  to  make  any 
necessary  modifications  to  the  CER  procedures. 

3)  Third,  a  possible  disadvantage  is  that  since  the  CER's  can 
be  expected  to  become  common  knowledge  in  the  defense  community,  there 
is  seme  danger  that  manufacturers  might  use  them  In  such  a  way  as  to 
affect  their  own  cost  proposals  in  an  undesirable  fashion.  However,  we 
have  not  yet  thought  through  the  possible  consequences.  If  more  often 
than  not,  this  leads  to  more  realistic  cost  proposals,  the  Defense  De¬ 
partment  will  probably  be  ahead. 


4)  Fourth,  the  use  of  CER's  does  not,  by  any  means,  eliminate 
uncertainty.  As  with  any  cost  estimating  procedure,  considerable  uncer¬ 
tainty  will  inevitably  be  present  on  estimates  of  future  systems.  How¬ 
ever,  use  of  a  statistical  approach  does  permit  the  computation  of  certain 
measures  of  uncertainty  -  the  confidence  or  prediction  intervals.  Never¬ 
theless  I  have  not  listed  this  as  a  major  advantage  for  statistical  CER's 
since: 

a)  I  am  not,  myself,  certain  that  these  predictions,  though 
useful,  have  the  meaning  they  purport  to  have.  Die  applicability  of  the 
computed  statistical  probability  statements  to  the  aircraft  program  being 
costed  is  not  clear. 

b)  With  any  estimation  technique,  some  quantitative  measure 
of  uncertainty  can,  and  should,  be  provided  even  if  subjectively  arrived  at. 

A  major  omission  to  the  program  of  this  symposium  is  the  absence 
of  a  paper  devoted  solely  to  the  subject  of  providing  quantitative  informa¬ 
tion  on  the  magnitude  of  the  uncertainty  of  particular  coat  estimates. 

5)  Finally,  it  ia  often  necessary  to  provide  estimates  of  pro¬ 
grams  whose  characteristics  are  outside  the  range  of  the  sample  data 
used  to  derive  the  CER's.  Statistical  theory  has  little  to  say  as  to 
whether  the  same  trends  reflected  in  the  cample  data  will  also  hold  be¬ 
yond  the  range  of  the  data.  However,  this  again  is  a  problem  cornu  to 
all  present  cost  estimation  techniques.  At  least  with  the  statistical 
CER,  past  trends  are  identified.  Nevertheless,  the  cost  estimator  should 
of  course  always  be  very  cautious  when  using  CER's  to  extrapolate  far 
beyond  the  range  of  the  sample  data. 
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